System and method for storing a candidate report

ABSTRACT

Extensive databases of well-annotated reports are important knowledge sources in medical workflows, such as in radiology. These are typically accessed by the healthcare professional looking for reports similar to a current case being evaluated. However, filling and maintaining such databases requires considerable effort. A system is provided for storing a candidate report, comprising a searcher configured to retrieve one or more queries from a query database; to retrieve the candidate report from a user input; to execute the one or more queries on the candidate report to determine the relevance of the candidate report, and to store the candidate report in the report database if the relevance exceeds a predetermined value. The invention provides a check, prior to storing, to evaluate whether a candidate report would actually be a worthwhile addition to the database. If not, it is by default not added. The healthcare professional is alerted when reporting on a case, so that superfluous cases are identified and left out of the database.

FIELD OF THE INVENTION

The invention relates the field of medical reports, and in particular toa system and method for storing a candidate report.

BACKGROUND OF THE INVENTION

Diagnostic reports generated as a result of medical procedure contain awealth of information, comprising diagnostic imaging results, clinicalhistory and annotations by healthcare professionals.

Radiologists typically process up to thirty cases per hour. When facinga challenging case, radiologists turn to information sources to completethe diagnosis. These information sources may include scientificpublications, but most frequently include previous cases. For thisreason, exemplary and interesting cases are collected in databases bythe radiologists themselves. Such databases may take many forms: localfolders on a computer system, accessible to individual users or tomultiple users, PACS systems, a reference case manager like mypacs.net(www.mypacs.net).

These databases comprise many reports, by far exceeding the number ofcases that any single person can recall. Even a personal folder,populated by one healthcare professional such as a radiologist, willtypically grow beyond the point where its user can recall its fullcontent. The problem is compounded by the access to databases ofmultiple users. For example, over 1 million diagnostic reports weregenerated in 1991-1996 by the Department of Radiology alone, and storedin the radiology information system (RIS) of the University of TexasMedical Branch. The ability to search the databases and retrieverelevant matches therefore becomes increasingly important. As medicalimaging becomes more affordable, and the diversity of diagnosticmodalities and therapeutic treatments increase, the amount of data beingstored increases, and the problem becomes even more critical.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a system and method forstoring a candidate report.

The invention is defined by the independent claims. Advantageousembodiments are defined in the dependent claims.

According to a first aspect of the invention, a system is provided forstoring a candidate report, comprising:

a report database configured to store and provide reports;

a user input configured to provide the candidate report;

a query database configured to store and provide queries, wherein thequeries are suitable for searching in the report database;

a searcher configured to:

-   -   retrieve one or more queries from the query database;    -   retrieve the candidate report from the user input;    -   execute the one or more queries on the candidate report to        determine the relevance of the candidate report, and    -   store the candidate report in the report database if the        relevance exceeds a predetermined value.

The invention is based upon the insight that the retrievability ofreports by the healthcare professional may be made more efficient byreducing the proportion of low-quality reports in the database. Afurther insight is that it may be even more efficient to evaluate areport for inclusion in the database before it is included.

In some cases, this may also reduce the number of reports in thedatabase. This is completely opposite to the conventional approach,which strives to improve the retrievability of reports by more complexsearch algorithms to search through all reports.

The invention provides a check, prior to storing, to evaluate whetherthe candidate diagnostic report would actually be a worthwhile additionto the database. If not, it is by default not added. The healthcareprofessional is alerted when reporting on a case, so that superfluouscases are identified and left out of the database. In this way, thequality of the database reports is maintained, so that future retrievalsby the same user or different users, are more efficient and accurate.

The check is performed by using a database of previous queries,selecting one or more queries, and executing the queries on thecandidate report. Based upon the results, the relevance of the candidatereport is determined. This relevance is compared to a predeterminedvalue, which represents a threshold for inclusion in the database. Ifthe candidate report exceeds this value, the system adds the candidatereport to the report database. In other words, the system determineswhether the candidate report would have been retrieved from the reportdatabase if it had been part of the database when the one or morequeries were performed.

An additional advantage is a saving in time for the healthcareprofessional. In many cases, the healthcare professional must spendextra time to prepare a report for storage and to increase the chancesof future retrieval. However, the professional's time is limited andwriting a basic report takes considerably less time than writing a fullreference report. This has been estimated to be approximately a 20%/80%relationship between writing a basic report and writing a full report.The efficiency is improved by directing the professional to spend themost time on worthwhile reports, and less time on less-worthwhilereports.

A further advantage is the incentive to spend the extra time on thereport if the invention indicates prior to storage that the reportappears worthwhile to the rest of the healthcare community. Thecontribution of reports, and the appreciation for the reports expressedby the invention, provides immediate feedback to the professional abouthis/her performance, and provides a source of motivation.

The difference between a full report and a basic report depends on thetype of report. For example, a radiologist wishing to create a referencereport may add supplementary information, such as differential diagnosesor references to publications. The radiologist may also improve thechances of retrieval based upon knowledge of the retrieval search enginealgorithm, such as adding a selection of appropriate keywords,classifications, standardizing language, standardized layout withstandardized headings.

In another aspect of the invention, the searcher may be furtherconfigured to select the one or more queries for retrieval from thequery database, based upon a parameter selected from the groupconsisting of: queries relating to a specific imaging modality, queriesperformed by a specific individual, queries performed by a specificgroup of individuals, queries performed by a specific healthcarespecialism, queries performed from a specific location, queriesperformed over specific periods of time, queries which produced nomatches when last performed, a specific number of queries, queriesrelating to a specific anatomical feature, queries with a specificclassification, and any combination thereof.

To reduce the required computational power or to increase efficiency,the queries used to check the report prior to possible storage may belimited based upon context of the query, such as only those queriesperformed by the radiological department, or a healthcare professionalmay only be interested in his/her own queries. Other possibilities whichmay be advantageous are the queries of the last few months or the lastso-many queries performed.

It may be particularly advantageous to use queries which previouslyproduced no matches—if the candidate report matched to a high-enoughdegree, this would indicate that the report is an extremely valuableaddition.

Of course, in practice, the healthcare professional may use acombination of these in an iterative fashion. The skilled person willalso realize that such parameters may be performed successively, andthat a composite relevance or importance factor may be derived fromthese multiple determinations.

In a further aspect of the invention, the searcher is configured todetermine the relevance of a report, based upon a parameter selectedfrom: the degree of the match between the one or more queries and thereport, the number of queries producing a match, the number of uniquequeries producing a match, or any combination thereof.

Methods of the determination of relevance are well-known in the art, sothe invention is not limited to any particular definition. As therelevance of the candidate document is determined to accommodate futureretrievals, it may be advantageous to determine the relevance of thecandidate report in a comparable way to the relevance determination madewhen the queries were originally executed by users attempting toretrieve reports from the report database.

In another aspect of the invention, the system further comprises adisplay configured to provide a representation to the user of therelevance of the candidate report.

In this way, the user who has a report available receives directfeedback about its value to the database.

According to a further aspect of the invention, the searcher is furtherconfigured to:

retrieve a multiplicity of reports from the report database;

execute the one or more queries on the multiplicity of reports todetermine a match list for each query, wherein the match list comprisesthe relevance values of the members of the match list;

determine the predetermined value, based upon relevance values of thematch list.

It may be advantageous to create match lists for queries. By executingthe query for a large number of reports in the database, a match list ofreports with the most relevant relevance values may be compiled for eachquery. The relevance values in the match list then form the basis fordetermining the threshold for including the candidate report in thedatabase.

The members of the match list may be determined in any appropriate wayknown in the art, such as by selecting the relevance values of reportsexceeding a second predetermined value, or by selecting a predeterminednumber of the highest relevance values of reports.

The searcher may then be further configured to store the match list inthe query database associated with the corresponding query, and retrievethe match list associated with the one or more queries from the querydatabase.

It may be advantageous to compile the match lists prior to providing thecandidate report, so that the computational power required is reduced.This compilation may also be performed at regular intervals. The matchlists may be stored in any location, but it may be most convenient tostore them in the query database.

As an example, the predetermined value of the threshold for storing thecandidate report may be determined by ranking the match list, based uponthe relevance of the members of the match list, and determining thepredetermined value based upon a position in the ranked match list.

Determining the relevance of the candidate report may then be simplifiedby comparing its relevance to a fictitious position in the ranked list.If the candidate report would have been higher than a predeterminedposition in the ranked list, the candidate report is considered valuableenough to be placed in the report database. This simplifies theconsideration of the relevance of the candidate report.

The searcher may also be configured to store the predetermined value inthe query database associated with the corresponding query, and retrievethe predetermined value associated with the one or more queries from thequery database.

This may be implemented by providing or determining a threshold valuefor each query. This may be advantageous because it requires a low levelof processing power when determining whether to store the candidatereport or not.

The match list may further comprise a report identification of themembers of the match list. This may be advantageous as it may be used bythe user to locate reports which are considered more relevant than thecandidate report. This may be useful if the user requires more feedbackabout why the candidate report is not ranked higher. Similarly, the usermay wish to locate reports considered less relevant than the candidatereport.

It may be advantageous to include a display in the system, configured toprovide a representation to the user of the relevance of the candidatereport. The user is then informed about how worthwhile the candidatereport is considered to be.

This may also be advantageous when the user wishes to add annotations ormake other changes to increase the relevance. It is also envisioned thata dynamic relevance be provided so that as changes are made, therelevance is displayed immediately providing an efficient method for theprofessional to optimize the future chance of retrieval.

In a still further aspect of the invention,

the user input is further configured to provide a request by the user tostore the candidate report in the report database, and

the searcher is further configured to store the candidate report in thereport database in response to the user request.

It is envisioned that the system is provided with a predeterminedthreshold for relevance, so that once the user provides the report tothe system, it is automatically stored if the relevance exceeds thepredetermined threshold. However, it may be advantageous to always givestorage confirmation to the user, or to provide the user with means todetermine that the report is always stored, irrespective of therelevance. This may be useful when reports in a new area or newdiscipline are being created for which there are no earlier queries.

The system of the invention in any of its embodiments may be comprisedin a workstation or a medical imaging acquisition apparatus.

In another aspect of the invention, a method is provided for storing acandidate report, the method comprising:

providing a report database;

providing the candidate report;

providing a query database, wherein the queries are suitable forsearching in the report database;

retrieving one or more queries from the query database;

executing the one or more queries on the candidate report to determinethe relevance of the candidate report, and

storing the candidate report in the report database if the relevanceexceeds a predetermined value.

The method may further comprise:

retrieving a multiplicity of reports from the report database;

executing the one or more queries on the multiplicity of reports todetermine a match list for each query, wherein the match list comprisesthe relevance values of the members of the match list; and

determining the predetermined value based upon relevance values of thematch list.

A computer program product is also envisioned for carrying out themethod of the invention, when loaded and run on a computer.

It will be appreciated by those skilled in the art that two or more ofthe above-mentioned embodiments, implementations, and/or aspects of theinvention may be combined in any way deemed useful.

Modifications and variations of the image acquisition apparatus, of theworkstation, of the system, and/or of the computer program product,which correspond to the described modifications and variations of themethod, can be carried out by a person skilled in the art on the basisof the present description.

It will also be obvious to the skilled person that the invention doesnot need to be limited to reports generated by radiologists. It may beused with reports comprising any kind of medical data, although it maybe advantageous for reports comprising medical imaging data. These mayrequire considerable storage capacity, and reducing the number ofreports stored may also reduce equipments costs.

This imaging data may be acquired by any imaging modality, such as X-rayImaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI),Ultrasound (US), Positron Emission Tomography (PET), Single PhotonEmission Computed Tomography (SPECT) and Nuclear Medicine (NM).

It is also envisioned that the invention may be used for any informationcontent to be accessed by a database, such as web pages being compiledfor addition to the Internet. Search engines are known in the art forsearching the Internet, and producing results to a query based uponrelevance. In that case, the candidate report may be considered to bethe web page as it is being created and the Internet is the reportdatabase. It is known that some Internet search engines maintain adatabase of queries, but the queries are not used to indicate theimportance of the web page prior to uploading to the Internet.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention are apparent from and will beelucidated with reference to the embodiments described hereinafter.

In the drawings:

FIG. 1 shows the system according to the invention,

FIG. 2 depicts the method according to the invention, and

FIG. 3 depicts a possible representation to the user on a display.

The Figures are purely diagrammatic and not drawn to scale. Particularlyfor clarity, some dimensions are exaggerated strongly. Similarcomponents in the Figures are denoted by the same reference numerals asmuch as possible.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A system 300 for storing a candidate report 370 is depicted in FIG. 1.The system 300 comprises:

a report database 360 configured to store and provide reports. These maybe local folders comprised in the system 300, accessible to individualusers or to multiple users. The report database may also be an on-linesystem, such as a PACS system or a reference case manager likemypacs.net, which may be made available by providing the system 300 withan appropriate network connection. The database may typically also allowdeletion of and annotations to the reports stored;

a user input 330 configured to provide the candidate report 370.Typically, the user input 330 provides for interaction with the systemin any form known in the art. For example, as icons, thumbnails, menus,and pull-down menus. The user input 330 may also comprise a keyboard,mouse, trackball, pointer, drawing tablet or the like. The user input330 may also provide speech recognition capabilities. As the user input330 provides a candidate report 370, typically the user input 330 willcooperate with a processor 320 to provide the report handling functions,such as editing and word-processing facilities;

a query database 350 configured to store and provide queries, whereinthe queries are suitable for searching in the report database 360. Itmay be convenient to locate the query database 350 with the reportdatabase 360, as the queries are associated with the reports, and arecompiled from queries to the report database 360. However, any locationis envisioned, such as locally or on-line via an appropriate networkconnection;

a searcher 340 configured to:

-   -   retrieve one or more queries from the query database 350;    -   retrieve the candidate report from the user input 330;    -   execute the one or more queries on the candidate report 370 to        determine the relevance of the candidate report 370, and    -   store the candidate report 370 in the report database 360 if the        relevance exceeds a predetermined value.

The searcher 340 may be implemented in any appropriate way known to theskilled person, such as a processor provided with connections to thedatabases 350, 360.

The queries in the query database 350 are suitable for searching in thereport database 360. The nature of the queries may be of any appropriatetype known to the person skilled in the art, and depends upon the natureof the reports and the type of users. The query also depends upon theway in which the searcher 340 is configured to execute the query todetermine the relevance of the candidate report 370.

For example, the article “MoSearch: A radiologist-friendly tool forfinding-based diagnostic report and image retrieval”, by Ramaswamy,Patterson, Yin, Goodacre (RSNA 1996); Radiographics 1996, 16, pages923-933, discloses a software package that allows radiologists toconduct sophisticated real-time searches of diagnostic reports on thebasis of patient characteristics, modality used, anatomy examined, andimaging findings and to easily review, refine, and output the results.It was designed and implemented in a large academic hospital. A notablefeature of this system of the prior art is the use of synonym-matchingand syntactic cues, which allows it to identify findings within the textof a diagnostic report much

more accurately than a simple keyword search can. This type of systemcan be easily

and inexpensively implemented on a personal-computer.

Described in this article are also other techniques known to the skilledperson, such as coding the reports, structured report entry according toa prescribed format, natural language processing of the text in thereports, simple or complex keyword searches.

The skilled person will also be familiar with other search algorithmsused by Internet search engines to determine relevance, such aswww.google.com and www.yahoo.com, where the relevance is translated to aranked list of hits.

It may therefore be advantageous for the skilled person implementing theinvention to define the queries, the reports together with the algorithmfor determining relevance, so that the requirements of the users of thesystem may be taken into account.

FIG. 3 depicts a possible implementation of the representations whichmay be provided to the user on a display 310, which may be comprised inthe system 300. The display comprises a representation of the candidatereport 370 and a representation of the relevance 380 of said candidatereport.

Many such representations are possible. For example, the candidatereport 370 may be made available as a series of tabs 260 for navigation.Such tabs 260 may include History, Findings, DDx, Diagnosis, Discussion,References and Comments. Generally, such a report may comprise imagingresults 270 and some explanatory text 250. It may be advantageous forthe processor 320 to be configured so as to provide word processingfacilities, so that the explanatory text 250 may be edited.

The representation of the relevance 380 may take any form known in theart. For example, the skilled person will be aware of relevance resultsexpressed as a percentage, conventionally 0 to 100%. FIG. 3 gives a moreschematic representation 380, comprising a level indicator 240 inside abox 220. The level indicator 240 is a shaded bar representing the valuesbetween 0 and the percentage relevance of the candidate report 370. Thisshaded bar 240 is overlaid onto the unshaded box 220 representing therelevance levels 0 to 100%.

Optionally, a control 210 may be provided for the user to request adetermination of the relevance of the candidate report 370. It is alsoenvisioned that such a control is integrated into a word-processingpackage, so that when a candidate report 370 is saved in a bufferfolder, the relevance is automatically calculated and displayed 380.

Optionally, other controls 290 may be available to provide otherconvenient functions to the user, such as allowing the user to specifyfurther details for the method, or to overrule determinations of thesystem.

The system is configured for performing the method 500 according to theinvention, which is depicted in FIG. 2. The method comprises:

providing 560 a report database 360;

-   -   providing 570 the candidate report 370;

providing 550 a query database 350, wherein the queries are suitable forsearching in the report database 360;

retrieving 541 one or more queries from the query database 350;

executing 542 the one or more queries on the candidate report 370 todetermine the relevance of the candidate report 370, and

storing 543 the candidate report 370 in the report database 360 if therelevance exceeds a predetermined value.

During use of the system, the candidate report 370 is provided 570 bythe user input 330.

One or more queries are retrieved 541 from the query database 350. Thenumber of queries retrieved 541 may be anywhere between one and theentire contents of the query database. The selection of the queries tobe retrieved may be determined by the skilled person, based upon therequirements of the user and the computational power available. Forexample, the queries retrieved may be those relating to:

a specific imaging modality only, such as reports comprising MRIresults. In practice, a professional may wish to evaluate images madewith more than one modality, for example a radiologist may wish to lookat X-ray images as well as MRI and CT scans; therefore, queries relatingto one or more specific modalities may be retrieved;

queries performed by a specific individual, such as the user himself ora recognized expert in the field;

queries performed by a specific group of individuals, such as thetrainees in the radiology department;

queries performed by a specific healthcare specialism, such as allradiologists;

queries performed from a specific location, such as a particularhospital or a particular department;

queries performed over a specific period of time, for example in thelast 3 months;

-   -   queries which produced no matches when last performed, because a        candidate report which scored a high-relevance for such a query        would be a particularly valuable addition to the database;

a specific number of queries, such as the last 1000 queries or the mostpopular 1000 queries;

queries relating to a specific anatomical feature, such as brain-head,neck-spine;

queries with a specific classification, such as prototypical,borderline, difficult.

It will be apparent to the skilled person that these may be used incombination, or in succession, or even as part of an iterativeevaluation. Such an algorithm may be provided automatically, or the usermay be allowed to indicate via the user input 330 personal selections ofqueries.

The queries retrieved 541 are then executed 542 on the candidate report370 to determine its relevance. The relevance 380 is typically thendisplayed on the display 310.

For example, the relevance may be determined 542 based upon the degreeof the match between the queries retrieved and the report, the number ofqueries producing a match, the number of unique queries producing amatch, or any combination thereof. Typically, an appropriate algorithmfor determining relevance will be selected by the skilled person, andthis algorithm will be used when queries are executed to retrievereports from the database 360. It may be advantageous to use the samealgorithm to determine the relevance of the candidate report 370,although this is not essential. The skilled person will be able to tunethe algorithms to achieve the desired efficiency of operation.

In a fully automatic system, the relevance of the candidate report 370is compared by the searcher 340 to a predetermined value, and if therelevance exceeds this value, the searcher 340 stores 543 the candidatereport 370 in the report database 360.

The predetermined value may also be represented on the display, forexample as a line 230 at an appropriate position in the unshaded box220. This provides a simple feedback to the user of the relativerelevance as being the difference between the extent of the shaded bar240 and the required relevance 230 for inclusion.

A highly-automated system is also envisioned, in which the relevance ofthe candidate report 370 is represented 380 on the display 310, butstorage is not automatic. A control 290 is provided so that the userrequests the searcher 340 to store the candidate. This may beadvantageous in those cases where:

the user wishes to add the candidate report 370 to the database 360without considering the determined relevance. For example, if a userwishes to anticipate future queries, or if the candidate report 370 isrelated to a new field or discipline for which no queries would bepresent in the query database 350;

the user wishes to be provided with a dynamic representation 380 of therelevance of the candidate report 370. In this way, the user may editthe candidate report 370 by adding content to increase the relevance,such as annotations, keywords, references, links. This would be mostefficient when the user is provided with some understanding of the wayin which the relevance algorithm determines the relevance. Determination542 of the relevance may be automatic, or may be requested by the userusing the control 210 provided; or

the user wishes to be provided with a dynamic representation 380 of therelevance of the candidate report 370. If the relevance is high enough,the user may edit the candidate report 370 by adding content to enrichthe contents and to make it more useful to other users of the system.Examples of content that may be added include: an appropriate title,relevant facts from the patient's history, an overview of findings,differential diagnoses (DDx), the eventual diagnosis, discussion,references and links to publications, links to other relevant reports.Determination 542 of the relevance may be automatic, or may be requestedby the user using the control 210 provided. This provides an incentiveto the user to invest the extra effort into increasing the quality ofthe report for future retrieval by other users.

A further control 290 may be provided enabling the user to request thesearcher 340 not to store the candidate report, irrespective of thedetermined relevance. This may be advantageous in those cases where theuser wishes to postpone the work on the report, or simply to exit themethod.

The predetermined value which represents the threshold for inclusion inthe report database may be a single value for all candidate reports 370.However, it may be advantageous for the skilled person to be able tooptimize the system by providing different predetermined values whichthe searcher 340 may utilize under different circumstances.

For example, the predetermined value may be related to a specificimaging modality, a specific individual, a specific group ofindividuals, a specific healthcare specialism, a specific location, aspecific anatomical feature, a specific classification, or anycombination thereof. It is also envisioned that the predetermined valuemay be dynamic to allow the skilled person to control the flow ofreports in the database—for example, it may be advantageous to set thethreshold relatively low when a new database 360 is created to quicklygrow the database 360, and then raise the threshold as the database 360becomes full.

The predetermined value may also be determined based upon the currentcontents of the report database 360. The searcher 340 may be furtherconfigured to retrieve a multiplicity of reports from the reportdatabase and to execute the retrieved queries on these reports todetermine a list of matches for each query. In other words, the searcher340 executes each of the retrieved queries on the report database 360 tosee what the hits are. This list of matches may have no members if nohits are obtained, or it may include all reports in the report database360 if the query is very generic. Typically, the number of members willbe somewhere between these extremes, and may be limited in anyappropriate way, such as only the relevance values exceeding a secondpredetermined value, or a predetermined number of the highest relevancevalues such as the top ten.

The match list comprises the relevance values of the hits. Theserelevance values may then be used to determine the predetermined valuein any appropriate way known to the skilled person. For example, theminimum relevance, the maximum relevance, a statistical operation suchas average relevance, or a weighted average.

The determination of a matched list for each of the retrieved queriesmay be performed immediately prior to determining the relevance of thecandidate report 370. This may be computationally intensive, andtherefore it may be advantageous to determine the matched list for eachquery at an earlier time.

For example, the match lists may be determined during off-peak times,and the predetermined values stored at some convenient location, such asin the query database 350. The searcher 340 must then be furtherconfigured to store and retrieve the predetermined value for each of theretrieved queries.

It is also envisioned that the match list for each query is stored. Thismay be advantageous in those cases where the threshold for storage isdetermined by a statistical operation in which the match list includingthe candidate report is compared to the match list without the report,or where the match list is ranked and the relevance is indicated as apotential position in this ranked list. For example, if for each querythe top-ten most relevant hits are stored, then the threshold forstoring the candidate report is that the report would be positioned inthe top-five if added to the database 360.

Alternatively, in those cases where the user is permitted to requeststorage of the candidate report 370, the match list may be representedon the display 310 with an indication of where the candidate report 370would appear in a ranked list. This is a particularly intuitiverepresentation, allowing the user to request storage where appropriate.

It may be further advantageous to provide the user with further controls290 to influence the future retrieval of the candidate report 370, suchas:

a copy relevance control 290 to copy keywords or annotations from areport corresponding to a relevance in the match list higher than therelevance of the candidate report 370. For example, if the user wishesto ensure that the report is always retrieved in response to a query,the match list may also comprise identifications of the reportscorresponding to the relevance values. The searcher 340 is thenconfigured to store and retrieve these report identifications comprisedin the match list, and the display 310 is configured to display the listto the user. By selecting the copy relevance control 290 and an entry inthe list, the searcher copies the appropriate keywords and annotationsfrom this entry in the list to the candidate report 370, giving thecandidate report 370 at least the same relevance value. Alternatively,the same control may be used to reduce the relevance of the candidatereport 370 for one or more queries;

a retrieval alert control 290 so that the user is alerted when thecandidate report 370 is retrieved by another user from the database 360after it has been added to the database 360. This also provides an extraincentive to the user as he/she receives feedback that the report isactually being retrieved by future queries;

a report retrieval control 290 so that the user may select a reportalready present in the database 360, and retrieve it so that it becomesa new candidate report 370. This allows a user to improve existingreports to increase their relevance, or to retrieve older candidatereports 370 which were stored and to review the change in relevanceexperienced over a period of time; and

a query alert control 290 so that the user may request the system tomonitor one or more queries. The searcher 340 may be configured tomonitor the match lists for the selected queries, and to send an alertto the user if a new hit is added to the list. This may be particularlyadvantageous for queries which return no hits at a particular moment intime. The skilled person will recognize this concept from the GoogleAlert service.

The skilled person, provided with the details of the method disclosed,will be able to implement numerous systems for performing the method, inaddition to the systems disclosed in this application. Typically, such asystem will comprise a computer, and the skilled person will be able toassign the function to a combination of hardware and software, andconsequently implement a computer program to carry out these methodswhen loaded and run on the computer.

A user may use a workstation to perform interactions, for example duringthe preparation of the medical report, the annotation of such a reportand the confirmation that the report is to be stored. The workstationmay then comprise a processor 320 to assist in performing the userinteractions and in performing the method of the invention. Theworkstation may then comprise the system according to the invention. Itis also envisioned that the system 300 may be comprised in a medicalimage acquisition apparatus.

It will be apparent to the skilled person that a plurality of thefunctions of the system may be implemented in parallel, so that the samehardware may be utilized to provide a number of report databases. Forexample, it may be advantageous to limit a particular report database toone particular discipline, such as radiology. This allows the skilledperson to optimize the query definition, the report structure and therelevance algorithm for radiologists. A second database for a seconddiscipline, such as pathology, may be provided in parallel, so that thesecond query definition, the second report structure and the secondrelevance algorithm may then be optimized for pathologists.

In the claims, any reference signs placed between parentheses shall notbe construed as limiting the claim. Use of the verb “comprise” and itsconjugations does not exclude the presence of elements or steps otherthan those stated in a claim. The article “a” or “an” preceding anelement does not exclude the presence of a plurality of such elements.The invention may be implemented by means of hardware comprising severaldistinct elements, and by means of a suitably programmed computer.

In the system claim enumerating a user input, a searcher, a reportdatabase, a query database, and a display, several of these means may beembodied by one and the same item of hardware.

The mere fact that certain measures are recited in mutually differentdependent claims does not indicate that a combination of these measurescannot be used to advantage.

1. A system (300) for storing a candidate report (370), the systemcomprising: a report database (360) configured to store and providereports; a user input (330) configured to provide the candidate report(370); a query database (350) configured to store and provide queries,wherein the queries are suitable for searching in the report database(360); a searcher (340) configured to: retrieve one or more queries fromthe query database (350); retrieve the candidate report from the userinput (330); execute the one or more queries on the candidate report(370) to determine the relevance of the candidate report (370), andstore the candidate report (370) in the report database (360) if therelevance exceeds a predetermined value.
 2. The system of claim 1,wherein: the searcher (340) is further configured to: retrieve amultiplicity of reports from the report database (360); execute the oneor more queries on the multiplicity of reports to determine a match listfor each query, wherein the match list comprises the relevance values ofthe members of the match list; determine the predetermined value, basedupon relevance values of the members of the match list.
 3. The system ofclaim 2, wherein: the searcher (340) is further configured to: store thepredetermined value in the query database (350) associated with thecorresponding query, and retrieve the predetermined value associatedwith the one or more queries from the query database (350).
 4. Thesystem of claim 2, wherein: the searcher (340) is further configured to:store the match list in the query database (350) associated with thecorresponding query, and retrieve the match list associated with the oneor more queries from the query database (350).
 5. The system of claim 4,wherein the match list further comprises: a report identification of themembers of the match list.
 6. The system of claim 2, wherein: thepredetermined value is determined by: ranking the match list, based uponthe relevance of the members of the match list, and determining thepredetermined value, based upon a position in the ranked match list. 7.The system of claim 2, wherein the members of the match list aredetermined by selecting reports with relevance values exceeding a secondpredetermined value, or by selecting a predetermined number of reportswith the highest relevance values.
 8. The system of claim 1, wherein:the searcher (340) is configured to determine the relevance of a report,based upon a parameter selected from: the degree of the match betweenthe one or more queries and the report, the number of queries producinga match, the number of unique queries producing a match, or anycombination thereof.
 9. The system of claim 1, wherein the systemfurther comprises: a display (310) configured to provide arepresentation (380) to the user of the relevance of the candidatereport (370).
 10. The system of claim 1, wherein: the user input (330)is further configured to provide a request by the user to store thecandidate report (370) in the report database (360), and the searcher(340) is further configured to store the candidate report (370) in thereport database (360) in response to the user request.
 11. The system ofclaim 1, wherein: the searcher (340) is further configured to select theone or more queries for retrieval from the query database (350), basedupon a parameter selected from the group consisting of: queries relatingto a specific imaging modality, queries performed by a specificindividual, queries performed by a specific group of individuals,queries performed by a specific healthcare specialism, queries performedfrom a specific location, queries performed over a specific period oftime, queries which produced no matches when last performed, specificnumber of queries, queries relating to a specific anatomical feature,queries with a specific classification, and any combination thereof. 12.A workstation or a medical imaging acquisition apparatus comprising thesystem according to claim
 1. 13. A method (500) of storing a candidatereport (370), the method comprising: providing (560) a report database(360); providing (570) the candidate report (370); providing (550) aquery database (350), wherein the queries are suitable for searching inthe report database (360); retrieving (541) one or more queries from thequery database (350); executing (542) the one or more queries on thecandidate report (370) to determine the relevance of the candidatereport (370), and storing (543) the candidate report (370) in the reportdatabase (360) if the relevance exceeds a predetermined value.
 14. Themethod of claim 13, wherein the method further comprises: retrieving amultiplicity of reports from the report database (360); executing theone or more queries on the multiplicity of reports to determine a matchlist for each query, wherein the match list comprises the relevancevalues of the members of the match list; and determining thepredetermined value, based upon relevance values of the members of thematch list.
 15. A computer program product for carrying out the methodof claim 13, when loaded and run on a computer.